Dynamic Test Generation to Find Integer Bugs in x86 Binary Linux Programs

Recently, integer bugs, including integer overflow, width conversion, and signed/unsigned conversion errors, have risen to become a common root cause for serious security vulnerabilities. We introduce new methods for discovering integer bugs using dynamic test generation on x86 binaries, and we describe key design choices in efficient symbolic execution of such programs. We implemented our methods in a prototype tool SmartFuzz, which we use to analyze Linux x86 binary executables. We also created a reporting service, metafuzz.com, to aid in triaging and reporting bugs found by SmartFuzz and the black-box fuzz testing tool zzuf. We report on experiments applying these tools to a range of software applications, including the mplayer media player, the exiv2 image metadata library, and ImageMagick convert. We also report on our experience using SmartFuzz, zzuf, and metafuzz.com to perform testing at scale with the Amazon Elastic Compute Cloud (EC2). To date, the metafuzz.com site has recorded more than 2, 614 test runs, comprising 2, 361, 595 test cases. Our experiments found approximately 77 total distinct bugs in 864 compute hours, costing us an average of $2.24 per bug at current EC2 rates. We quantify the overlap in bugs found by the two tools, and we show that SmartFuzz finds bugs missed by zzuf, including one program where SmartFuzz finds bugs but zzuf does not.